模拟器为因果效应估计制作独特的基准，因为它们不依赖于无法验证的假设或干预现实世界的能力，但往往太简单，无法捕获实际应用的重要方面。我们提出了Alzheimer疾病的模拟器，旨在建模医疗保健数据的复杂性，同时实现因果效应和政策估算的基准。我们将系统拟合到阿尔茨海默病神经影像倡议（ADNI）数据集和地面手工制作组件，从比较治疗试验和观察治疗模式的结果中。模拟器包括改变因果推理任务的性质和难度，例如潜在变量，效果异质性，观察到的历史长度，行为策略和样本大小的参数。我们使用模拟器比较平均和条件治疗效果的估计。

High-dimensional data can often display heterogeneity due to heteroscedastic variance or inhomogeneous covariate effects. Penalized quantile and expectile regression methods offer useful tools to detect heteroscedasticity in high-dimensional data. The former is computationally challenging due to the non-smooth nature of the check loss, and the latter is sensitive to heavy-tailed error distributions. In this paper, we propose and study (penalized) robust expectile regression (retire), with a focus on iteratively reweighted $\ell_1$-penalization which reduces the estimation bias from $\ell_1$-penalization and leads to oracle properties. Theoretically, we establish the statistical properties of the retire estimator under two regimes: (i) low-dimensional regime in which $d \ll n$; (ii) high-dimensional regime in which $s\ll n\ll d$ with $s$ denoting the number of significant predictors. In the high-dimensional setting, we carefully characterize the solution path of the iteratively reweighted $\ell_1$-penalized retire estimation, adapted from the local linear approximation algorithm for folded-concave regularization. Under a mild minimum signal strength condition, we show that after as many as $\log(\log d)$ iterations the final iterate enjoys the oracle convergence rate. At each iteration, the weighted $\ell_1$-penalized convex program can be efficiently solved by a semismooth Newton coordinate descent algorithm. Numerical studies demonstrate the competitive performance of the proposed procedure compared with either non-robust or quantile regression based alternatives.

We analyze Newton's method with lazy Hessian updates for solving general possibly non-convex optimization problems. We propose to reuse a previously seen Hessian for several iterations while computing new gradients at each step of the method. This significantly reduces the overall arithmetical complexity of second-order optimization schemes. By using the cubic regularization technique, we establish fast global convergence of our method to a second-order stationary point, while the Hessian does not need to be updated each iteration. For convex problems, we justify global and local superlinear rates for lazy Newton steps with quadratic regularization, which is easier to compute. The optimal frequency for updating the Hessian is once every $d$ iterations, where $d$ is the dimension of the problem. This provably improves the total arithmetical complexity of second-order algorithms by a factor $\sqrt{d}$.

Is it possible for a first-order method, i.e., only first derivatives allowed, to be quadratically convergent? For univariate loss functions, the answer is yes -- the Steffensen method avoids second derivatives and is still quadratically convergent like Newton method. By incorporating an optimal step size we can even push its convergence order beyond quadratic to $1+\sqrt{2} \approx 2.414$. While such high convergence orders are a pointless overkill for a deterministic algorithm, they become rewarding when the algorithm is randomized for problems of massive sizes, as randomization invariably compromises convergence speed. We will introduce two adaptive learning rates inspired by the Steffensen method, intended for use in a stochastic optimization setting and requires no hyperparameter tuning aside from batch size. Extensive experiments show that they compare favorably with several existing first-order methods. When restricted to a quadratic objective, our stochastic Steffensen methods reduce to randomized Kaczmarz method -- note that this is not true for SGD or SLBFGS -- and thus we may also view our methods as a generalization of randomized Kaczmarz to arbitrary objectives.

Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.

We study the convergence of several natural policy gradient (NPG) methods in infinite-horizon discounted Markov decision processes with regular policy parametrizations. For a variety of NPGs and reward functions we show that the trajectories in state-action space are solutions of gradient flows with respect to Hessian geometries, based on which we obtain global convergence guarantees and convergence rates. In particular, we show linear convergence for unregularized and regularized NPG flows with the metrics proposed by Kakade and Morimura and co-authors by observing that these arise from the Hessian geometries of conditional entropy and entropy respectively. Further, we obtain sublinear convergence rates for Hessian geometries arising from other convex functions like log-barriers. Finally, we interpret the discrete-time NPG methods with regularized rewards as inexact Newton methods if the NPG is defined with respect to the Hessian geometry of the regularizer. This yields local quadratic convergence rates of these methods for step size equal to the penalization strength.

由于免费的在线百科全书具有大量内容，因此Wikipedia和Wikidata是许多自然语言处理（NLP）任务的关键，例如信息检索，知识基础构建，机器翻译，文本分类和文本摘要。在本文中，我们介绍了Wikides，这是一个新颖的数据集，用于为文本摘要问题提供Wikipedia文章的简短描述。该数据集由6987个主题上的80K英语样本组成。我们设置了一种两阶段的摘要方法 - 描述生成（I阶段）和候选排名（II阶段）作为一种依赖于转移和对比学习的强大方法。对于描述生成，与其他小规模的预训练模型相比，T5和BART表现出了优越性。通过将对比度学习与Beam Search的不同输入一起应用，基于度量的排名模型优于直接描述生成模型，在主题独立拆分和独立于主题的独立拆分中，最高可达22个胭脂。此外，第II期中的结果描述得到了人类评估的支持，其中45.33％以上，而I阶段的23.66％则支持针对黄金描述。在情感分析方面，生成的描述无法有效地从段落中捕获所有情感极性，同时从黄金描述中更好地完成此任务。自动产生的新描述减少了人类为创建它们的努力，并丰富了基于Wikidata的知识图。我们的论文对Wikipedia和Wikidata产生了实际影响，因为有成千上万的描述。最后，我们预计Wikides将成为从短段落中捕获显着信息的相关作品的有用数据集。策划的数据集可公开可用：https：//github.com/declare-lab/wikides。

矩阵函数可用于重写光滑光谱约束的矩阵优化问题，因为在一组对称矩阵的集合中，不受限制的问题，然后通过立方规范化的牛顿方法求解。事实证明，矩阵函数的二阶链条规则身份可以计算高阶导数以实现立方规范化的牛顿，并为矩阵矢量空间的立方调节牛顿提供了新的收敛分析。我们通过在合成数据集和真实数据集上进行数值实验来证明我们的方法的适用性。在我们的实验中，我们制定了一个新的模型，以估算泰勒的M-估计器（TME）模型的精神估算公平和稳健的协方差矩阵并证明其优势。

预处理一直是优化和机器学习方面的主食技术。它通常会减少其应用于矩阵的条件数，从而加快优化算法的收敛性。尽管实践中有许多流行的预处理技术，但大多数人缺乏降低病数的理论保证。在本文中，我们研究了最佳对角线预处理的问题，以分别或同时分别或同时缩放其行或列来实现任何全级矩阵的条件数量的最大降低。我们首先将问题重新将问题重新制定为一个准凸出问题，并提供了一种基线一分配算法，该算法在实践中易于实现，其中每次迭代都包含SDP可行性问题。然后，我们建议使用$ o（\ log（\ frac {1} {\ epsilon}）））$迭代复杂度提出多项式时间潜在的降低算法，其中每个迭代均由基于Nesterov-todd方向的牛顿更新组成。我们的算法基于该问题的表述，该问题是von Neumann最佳生长问题的广义版本。接下来，我们专注于单方面的最佳对角线预处理问题，并证明它们可以作为标准双SDP问题配方，我们应用了有效的定制求解器并研究我们最佳的对角线预处理的经验性能。我们在大型矩阵上进行的广泛实验表明，与基于启发式的预处理相比，最佳对角线预处理在减少条件数方面的实际吸引力。

素描和项目是一个框架，它统一了许多已知的迭代方法来求解线性系统及其变体，并进一步扩展了非线性优化问题。它包括流行的方法，例如随机kaczmarz，坐标下降，凸优化的牛顿方法的变体等。在本文中，我们通过新的紧密频谱边界为预期的草图投影矩阵获得了素描和项目的收敛速率的敏锐保证。我们的估计值揭示了素描和项目的收敛率与另一个众所周知但看似无关的算法家族的近似误差之间的联系，这些算法使用草图加速了流行的矩阵因子化，例如QR和SVD。这种连接使我们更接近准确量化草图和项目求解器的性能如何取决于其草图大小。我们的分析不仅涵盖了高斯和次高斯的素描矩阵，还涵盖了一个有效的稀疏素描方法，称为较少的嵌入方法。我们的实验备份了理论，并证明即使极稀疏的草图在实践中也显示出相同的收敛属性。